Papers by Tomás Vergara Browne
Tracr-Injection: Distilling Algorithms into Pre-trained Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent efforts to characterize symbolic abilities of the transformer architecture have shown that the tasks that can be implemented in RASP are uncommon to learn from natural unsupervised data. |
| Approach: | They propose a programming language, called RASP, which can be directly compiled into transformer weights to implement these algorithms. |
| Outcome: | The proposed method improves out-of-distribution performance compared to baselines, indicating that indeed a more symbolic mechanism is taking place in the inner workings of the model. |
From Insights to Actions: The Impact of Interpretability and Analysis Research on NLP (2024.emnlp-main)
Copied to clipboard
| Challenge: | Interpretability and analysis (IA) research is a growing subfield within NLP . a criticism of this work is that it lacks actionable insights and therefore has little impact on NLP. |
| Approach: | They propose to quantify the impact of interpretation and analysis research on NLP . they use citation graphs and a survey to find out what is missing in IA research . |
| Outcome: | The proposed study shows that IA research is well-cited outside of IA and central in the NLP citation graph. |